Skip to content

util: add default_value_vec for defaults without LengthReadable#4507

Open
carlaKC wants to merge 1 commit intolightningdevkit:mainfrom
carlaKC:default-vec-macro
Open

util: add default_value_vec for defaults without LengthReadable#4507
carlaKC wants to merge 1 commit intolightningdevkit:mainfrom
carlaKC:default-vec-macro

Conversation

@carlaKC
Copy link
Contributor

@carlaKC carlaKC commented Mar 23, 2026

⚠️ Clauded and did not review this - just looking for a concept ack and I"ll clean it up properly !


In lightningdevkit/ldk-node#825, I ran into the orphan rule when trying to migrate from a single incoming/outgoing HTLC to a vec of HTLCLocator types to allow expressing multiple in/out HTLCs. This issue stems from the following:

  • We want to declare our own HTLCLocator type in LDK-Node, so that we can have niceties like a typed user_channel_id.
  • We don't want to break out of the impl_writeable_tlv_based_enum! macro for LDK-Node's Event type.
  • To be forwards compatible (use the legacy single-htlc fields), we should use default_value to fill the old fields when the new ones aren't present.

However:

  • default_value uses required under the hood, which expects the field to be LengthReadable.
  • We cannot implement LengthReadable for Vec<HTLCLocator> in LDK-Node due to the orphan rule (we don't own Vec<T>, even if T is our type).
    • In LDK, this is when we use impl_for_vec! to implement this.

This change introduces a new default_value_vec to our macro which uses required_vec (and thus WithoutLength) under the hood, so that we don't need LengthReadable. There are a few ugly workarounds (like using custom), but wanting to write a Vec with a default seems like a common enough one to justify a change to our macros.

@ldk-reviews-bot
Copy link

ldk-reviews-bot commented Mar 23, 2026

I've assigned @jkczyz as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@codecov
Copy link

codecov bot commented Mar 23, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.17%. Comparing base (1f0763f) to head (1a24f8f).
⚠️ Report is 18 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4507      +/-   ##
==========================================
- Coverage   86.18%   86.17%   -0.02%     
==========================================
  Files         160      160              
  Lines      107537   107562      +25     
  Branches   107537   107562      +25     
==========================================
+ Hits        92681    92690       +9     
- Misses      12229    12243      +14     
- Partials     2627     2629       +2     
Flag Coverage Δ
tests 86.17% <100.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

items: Vec<u32>,
}
impl_writeable_tlv_based!(DefaultValueVecStruct, {
(1, items, (default_value_vec, Vec::new())),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Might make sense to use a dummy default value here rather than the empty Vec to check this is actually what's used rather than Vec::default() in general?

@TheBlueMatt
Copy link
Collaborator

default_value uses required under the hood, which expects the field to be LengthReadable.
We cannot implement LengthReadable for Vec in LDK-Node due to the orphan rule (we don't own Vec, even if T is our type).
In LDK, this is when we use impl_for_vec! to implement this.

Hmm, I guess we could use custom for this, no? In both impls wrap with WithoutLength and call it a day?

I'm not a huge fan of yet more write variants, ideally we'd drop some, really, but I do see why this is useful, and its not much of an extension on what we already have, so if custom isn't super clean I'm fine with this.

@carlaKC
Copy link
Contributor Author

carlaKC commented Mar 24, 2026

Hmm, I guess we could use custom for this, no? In both impls wrap with WithoutLength and call it a day?

Yeah we can, it's just pretty ugly, worth having the option IMO.

Right now, use of `default_value` requires that the struct implements
`LengthReadable` itself. When trying to use `default_value` outside of
LDK for `Vec<T>`, your code will run into the orphan rule because it
does not own the trait `LengthReadable` or the type `Vec`.

There are various ugly workarounds for this (like using `custom`), but
wanting to persist a vec with a default value seems like a common enough
use case to justify the change.
@TheBlueMatt
Copy link
Collaborator

Yeah we can, it's just pretty lightningdevkit/ldk-node#825 (comment), worth having the option IMO.

Oof, yea, that's not great. Happy to see this move forward.

@carlaKC carlaKC marked this pull request as ready for review March 26, 2026 20:17
@ldk-reviews-bot ldk-reviews-bot requested a review from jkczyz March 26, 2026 20:18
@ldk-claude-review-bot
Copy link
Collaborator

No issues found.

I reviewed every macro arm in the diff across all 7 dispatch macros (_encode_tlv, _get_varint_length_prefixed_tlv_length, _check_decoded_tlv_order, _check_missing_tlv, _decode_tlv, _init_tlv_based_struct_field, _init_tlv_field_var) and verified:

  • The encoding chain (default_value_vec -> required_vec -> WithoutLength -> required) correctly writes elements without a count prefix, always emitting the TLV even for empty vecs.
  • The decode path reads WithoutLength<Vec<_>> and wraps in Some, consistent with the Option<Vec<_>> field variable.
  • Default application in _check_decoded_tlv_order and _check_missing_tlv correctly sets Some($default) when the TLV is absent, and does not double-apply.
  • The unwrap() in _init_tlv_based_struct_field is safe because all code paths (decode or default-apply) guarantee Some.
  • Generic dispatch macros (_check_encoded_tlv_order, _decode_tlv_stream_match_check) handle default_value_vec via existing $fieldty: tt catch-all arms.
  • Proc macros (drop_legacy_field_definition, skip_legacy_fields) correctly pass through default_value_vec fields (they only drop legacy/custom).
  • Test hex values are correct ("00" = empty TLV stream, "0600040000002a" = type 0, length 4, u32 42).
  • The legacy-to-vec migration pattern works correctly with lazy default evaluation referencing the legacy field variable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

5 participants